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Sequential Diversity Imaging 

Inventor: P.obcrt A. Gonsalves* Wohum, Mass. 

Abstract 



Diversity imaging uses two or more diverse images of an object to enhance the detected images 
aad/ ok to improve the image quality of an optical system, The method estimates both the 
uokucvm object and the parameters of any disturbances in the optical system. The estimated 
parameters can be used to control an adaptive optic in the opticad system so as to sharpen the 
image. In this invention we use toe-sequential images from such an optical sy stem and time 
sequential diversities to develop control signals for the adaptive optic We show that the 
required sequential diversities are the sequential changes in ehe adaptive optic. 
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BACKGROUND OF THE INVENTION 

The present invention relates to imaging through a i-andom medium such as the 
turbulent atmosphere or the slowly changing optics of a camera; arid how to combat the 
deleterious effects of that medium, in a real-time end an automatic way. We show how to 
determine the distorting wavefront caused by the medium and how to control m adaptive optic 
in the optical system to eliminate that wavefroat. 

Diversity imaging uses one or more of the images, each with a known diversity (phase, 
wavelength, or spatial shift), *o deduce both the unknown object and the parameters of the 
medium. Gonsalves and Devaney held the fust patent of this kind and the method was usad by 
Gonsalves and others to deduce the flaw in the Hubble Space Telescope, In the usual ^ 
embodiment of this method the diversity is a quadratic phase shift, which can be introduced by 
defocusiug the optical system. Love, et al„ Duncan, el aL and Paxma.n hold current patents 
which use phase diversity, 

Unlike the patents by Oonsalves and others this invention uses only the measured 
image sequence, itself, as dm. No additions! image, such as a second, defocuscd image is 
required. Successive images form the multiplicity of images required for diversity imaging 
and the changes to the adaptive optic are the diversities. 

Thb novel approach eliminates the need for any other mechanises, It applies to any 
adaptive optic and to any digital camera. The renovations are the use of sequential images as 
the diversity mages and the use of a sequential diversity processor to control, the adaptive 
optic. 

Further background references are listed below: 
References 

1. Jt V, Geieftberg and 0, Saxon, "Phase Determination from Linage and Diffraction 

Plane Pictures", Optic, 34,275 (1971). ^ ^ * « n/, 

2. R, OoBsaives, "Phase Retrieval from Modulus Data", I Opt. Soc. of Am, 66, 901 u*'&J- 

3. I Fienup, "Phase retrieval algorithms a comparison;' Ap. Opt., 21, ^2758 (1982) 

4. R. Gonsalves. "Phase Retrieval and Diversity in Adaptive Optics , Opt. Eng., 21, 8** {l 

5. B. Filcrbroek and D. Morrison, "Linear methods in phase retrieval/ SPIE Proc. 35 L 
90 (1982> 

6. WL Tcagite, "Deterministic phase retrieval: a Greeks function solution/ I Opt. Sac. of Am. 

73,1434(1983). _ _ . t r 7 ~ _ f 

7 R. Gonsatos, "Phase Retrieval by Differential Intensity Measurements , J, Opt. hoc. oi 

Am A, 4, 166(1987), 

8. R. Paxman and J. Fienup, "Optical misalignment sensing and image reconstruction using 
phase diversity/ J. Opt Soc. of America, A 5, 914 (1988; ^ 

9. E Readier e-t aL "A Simple Low-Order Adaptive Optics System For Near-Scared 
Application^ Publications of the Astronomical Society of the Pacific, 103, 131 (i99;), 

1 0, I E Graves « al, The University of Hawaii adaptive optics system: 111 The Wavcfront 
Curvature Sensor", SPIE Proc, 1542, 262 (1991). 

II R, Kendtick* D, Acton, A, Duncan, 'Phase-diversity wave-float sensor for imaging 
gv$rems," ApL Opt 33, 6533 (1994). . # 

12. k A. Voronstov and V. P. Sfootofc " Stochastic paral.^gradieru.descent ™ 
high-resoUitior. wave-ftont phase-distortion correction," J. Opt, Soc. Am, A, 1^2^ 



BRIEF DESCRIPTION OF THE DRAWINGS: 

Fig. 1 Is a combined block-pictorial representation of an imaging system which uses as 
adaptive optic and a sequential diversity processor, in accordance with the invention. 

Fig. 2 is a block diagram of the sequential diversity processor, showing the input 
images from a detector and the output control signals for the adaptive optic. 

Fig, 3 shows a computer simulation of the invention as it would apply to the imaging of 
a point source, like a star, imaged through a turbulent atmosphere. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

The detector seen in both Fig. 1 and in Fig. 2 is one in which the picture is captured as 
discrete picture elements - essentially, a digital camera. We propose no new detecting 
mechanism for the camera, digital cameras exist and arc improving in sensitivity and 
resolution at a dramatic pace. 

The adaptive optic (AO) shown in both Fig. 1 and in Fig. 2 is a high-resolution device 
which allow a wide range of correction mechanisms, such as Zernike polynomial fitting of a ^ 
complicated wavefront. It is similar to the sophisticated systems currently installed on national 
telescopes and ft has capabilities well beyond the automatic focussing and registration 
mechanisms in consumer camcorders. These sophisticated AO systems improve image quality 
by about a factor often. They will almost certainly appear in consumer products, after the cost 
of adaptive optics decrease and after simple, real-time control mechanisms are invented. 

The AO in these figures is used to cancel the wavefront distortion, W. Thus, to be 
effective, the optical system of Fig, 1 must sense the wavefront There are five well-known 
mechanisms for wavefront sensing, which we review here. The Five are as follows: 

1 . Dithering. This method continuously changes the adaptive optic and monitors 
the image quality of the observed image. The method was patented by O'Meara, who made 
slowly varying changes to each channel of the adaptive optic, modulated each channel, 
demodulated the image quality in each channel, and controlled the adaptive optic with the set 
of image qualities. A recent improvement, stochastic paraUel-giadient-deacent, has been 
proposed by Voronstov and Sivokon (Reference 12). They make random changes in all 
channels and, over time, determine which average changes improve the image quality. 

2. Shearing Interferometer. This method uses a reference beam to create an 
interference pattern which describes the unknown wavefront. Hardy was the first to patent a 
wavefront sensor based on this device. It has been used successfully in several astronomical 
installations. Its major drawback is the extra measuring equipment, including a laser-based 
interferometer, that has to be installed. 

3 Shack-Hartmann Sensor. This device uses an array of lenses to image small 
sec-dons of the wavefront onto a detector. Shifts in the small images ate caused by local tilts in 
the waveform. From these tilts one reconstructs the wavefront. Feinleib held the first patent, 
A major drawback is the requirement for the lenslet array and sensors. 
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4. Curvature Sensing. This uses two or more images, measured along the path of 
the optical system. The local curvature of the propagating wave is determined and it is 
propagated, by computer simulation, back to the aperture. The computer-generated phase 
controls an adaptive optic, The method was proposed by Roddier and related patents were 
awarded to GatTard, et. a! and Rafanelli, et aL A system of this type is in use on a telescope in 
the state cf Hawaii, References 6, 7, 9 and 10 describe systems of this kind. 

5 . Phase retrieval and diversity. Our invention is similar to these, in xhat they use 
digital imageiy , an adaptive optic, and multiple images. The method was first suggested by 
Gonsalves (Reference 2). References l-5 9 8 and 1 1 use phase retrieve anchor diversity. 

Many investor* have tried to use post -processing of a scries of images to enhance the 
composite. Stents to Rhodes fall into this category. Wbiie such techniques may be useful if 
the data are already captured *;id no adaptive optic was available during measurement, this 
inventor teaches, H If ycu want a good picture, take a good pietiue." This implies that it is far 
better to improve the image m its taking than to try to ixnpiove the image by computer 
processing, after the feet. 

Referring to Fig. I and Fig, 2, \ve use the following notation: 

Wfk) ~ Unknown distorting wavefiront at time k, 

Tfk) "* Phase pur on the AO at time k 

C(k) - Compensated phase to be estimated by a diversity algorithm 

- W0c) T0r) . (1; 

I(k) = Measured image at time k 

D(k) = Diversity phase. 

We consider a diversity algorithm where I(k-l) is the first image md I(k) is the 
diversity image, With this convention the di versity phase is the change in the AO phase from 
time k-l to time k. Thus, 

D(k) * T(k)-T(frl). (2) 

The phase diversity algorithm is set up to estimate C(k), the compensated phase at time 
k Call the estimate Q(k), Q{k) is, front equation (1), 

Q(k) - WJ(k) + T(k), (3) 

where Wi (k) is m estimate ofW(k), the unknown phase at time k. (We know T(kV so it need 
not be estimated.) At time k ^ 1 we would like to set the AO phase to the negative of the 
unknown *avefto«t at time k+1 . We do not know W at time k+1 but we do have an estimate 



of the wavefront at time k, namely WUk) Thi, ™» ^ a • 
updates are well within -1,1 W " s ^ be a estimate of W{k+ 1) if AO 
*m within J* ttme constant of the changing medium. Tl.us.weit 

T(k*l) = .wi(k) s 
which will tend to caned the wavefront distortion at k+1 , 
Solving (3) for Wl(k) and substituting it into (4), we have 

T(k+1) - . Q( k) +T(k) ( 
which implies 

T &) - -Q(k-1) +T(k-1). (5) 

Turning to the diversity phase, we put (5) into (2) to get 
D(k) -(-Q(k.l) + T(k.l)).T<k.l) 

= -Q(k-l). (6) 
This is the specification for the diversity phase at time k. 
Finally, we put (6) into (5) to get a new expression for the AO phase: 

T(k) = r(k-1) + D(k). (7) 

Jigorltt m ' ** diveisi * is ^ - of fe diversity 

These equations are expressed in block diagram form in Fi<r 2 Th* inm** » »k» 

compensated phase, the phase diversity is D(k); and a feedback loop calculates T(k) 

,» . . F * 3 sh ™s a computer simulation of 24 cycles of the sequential diversity imager 

uSS%?? 'iT* S0UrCC> T h aS a ^ ^ first observed of the stitS lower 
WR of Fig 3. The compensated image is directly above. The laatjair is in the upper rkrT 

a SIS™ r J charactenstic of 8 «* in-ted through the atmosphere, 
sta^ ir^t? ^ t ^ qUa1 ^' i$ Strehl mti0 ' 1 e " ,he P eak value of the aberrated 

^ 24 mia 8 es shown, the average Strehl ratio is 0 045 

^rh JnS SeqUemial 1Sity ima8ing techn ^ ue P"* Jutt,d the ima S« shown directly above 
each twmkbig image. The average Strehl ratio for the compensated images is 0 467 This is 
aaimprovemeniofaboutafiwtoroflO g Jmsis 
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• We n.ot« that in our simulation the time constant of the atmospheric disturbance i: abou 
eight images; that i< ? the tf&r image takes sbcut eight cycles to appear completely uncorrected 
with ii% earlier image Thus th° cycle time of the imager is, In this simulation, eight times 
shorter than the tina cyck of the atmosphere. The latter is quite variable, but a reasonable 
estimate is 8 njitli^orids. Thus, the sequential processor must measure the new image and 
calculate the new AO settings in about erne millisecond. 

When we- doubled the atmospheric time constant to 16 ms, the average Streh' ratio of 
the imager improved to 0,65, an increase of about 14 in the image qualit> , over the 
uncompensated .iiar images, When we halved the atmospheric time constant to 4 ms, the 
average Strchi ratio improved only to 0,29, an increase in image quality of a factor of about. 6. 

Fb.aU> , we allowed the ataosphwic time constant to vary irons 2 to 16 ids over the 24- 
image sequence and the average quality factor still improved - by a fector of 7.5. More 
significantly, when the time constant time was very small arid the processor was unable tc keep 
up with the rajviiy changing atmosphere, it now thekss recovered, without catastrophic 
failure. That is, the processor is self-con ecting. 
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Fig, 2 Details of the Sequential Diversity Processor 
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Fig, 3, Computer simulation of 24 cycles of Sequential Diversity Imaging 
First image pair is in the lower left, last is in the upper right 
Lower rows are uncompensated, upper rows are compensated 
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